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Abstract 


Recently there has been a considerable interest in the Technological Pedagogical Content Knowledge 
(TPACK) framework for effective technology integration (Koehler et al., 2013). Researchers have been 
measuring how effective their professional development efforts have been for the development of teachers’ 
TPACK (Graham et al., 2009; Guzey & Roehrig, 2009). So far research is based only on short-term 
courses, so longitudinal studies are needed to examine the TPACK development of teachers across time 
(Hoffer & Grandgenett, 2012; Koh & Sing, 2011). 

A two-year in-service training program of educational technology (60 ECTS), based on the TPACK 
framework, was developed at Tallinn University Haapsalu College to support in-service teachers to 
effectively embed ICT into their classroom teaching. The first group of teachers (n=20) enrolled in the 
program in January 2014. This paper reports some preliminary findings of a longitudinal action research, 
the main aim of which is to evaluate the impact of the in-service training program of educational 
technology on teachers’ TPACK. 

The key questions for this research were: how do teachers’ perceptions of their TPACK levels change after 
participating in the training program for a year; which of the seven knowledge domains of the TPACK 
framework develop more than others; what is the impact of different knowledge domains on teachers’ 
TPACK perceptions. 

The TPACK questionnaire developed by Schmidt et al. (2009) was used to measure in-service teachers’ 
self-assessments of their TPACK during the first meeting of the training program and at the end of the 
first academic year. 

The research results revealed increase in all the domains of the TPACK framework. There was significant 
development of the teachers’ TK and TPACK, but only limited growth in CK and PK. Strong positive 
correlations were found between TCK, TPK and TPACK in the post-course survey. 

Key words: CT, in-service teachers, professional development, TPACK. 


Introduction 


Many educational organizations point to the importance of training teachers to more 
effectively integrate technology in their classroom (International Society for Technology in 
Education, 2008; Partnership for 21st Century Skills, 2010). Yet, research implies that effective 
use of technology remains challenging for most teachers (Lemke, Coughlin & Reifsneider, 
2009; Koehler et al., 2013). 

The EST_IT@2018 report launched by the Estonian Development Fund lists the strengths 
and weaknesses of the Estonian educational system. One of the weaknesses mentioned is that 
teacher-training institutions do not ensure that their graduates possess the necessary ICT skills 
for teaching (Eesti Arengufond, 2010). In-service teachers also report feeling unprepared on 
how to use ICT in the classroom to support learning. ICT was the second highest area identified 
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as a *high development need’ by in-service teachers in the TALIS survey in 2013 (OECD, 
2014). 

Evidence shows that increasing professional development opportunities for teachers is 
an efficient way of boosting ICT use in teaching and learning, since it helps to build highly 
confident teachers (European Schoolnet, 2013). However, Keller et al. (2008) have pointed out 
that not all professional development opportunities are equally effective. Traditional one-time 
teacher training workshops and conferences do not seem to have a lasting impact on teachers’ 
practice (Carlson & Gadio, 2002; Enochsson & Rizza, 2009). Instead, continuous and sustained 
training is needed (Sardone & Devlin-Scherer, 2008). Lawless and Pellegrino (2007) also 
stress that high-quality professional development must be longer in duration, provide access 
to new technologies for teaching and actively engage teachers in meaningful activities for 
their individual contexts. Moreover, providing only technical skills training to teachers is not 
enough. Teachers also need professional development in the pedagogical application of those 
skills to improve their teaching (Carlson & Gadio, 2002). 

Recently there has been a considerable interest in the Technological Pedagogical Content 
Knowledge (TPACK) framework, which aims to describe the kinds of knowledge needed by 
a teacher for effective pedagogical practice in a technology-enhanced learning environment 
(Mishra & Koehler, 2006; Harris, Mishra & Koehler, 2009). The TPACK framework argues 
that effective technology integration for teaching specific subject matter requires understanding 
the relationships between technology, pedagogy, and content. 

There are altogether seven knowledge domains within the TPACK framework (Figure 
1): 

1) technological knowledge (TK) - knowledge of various technologies, such as 
smartphones, interactive whiteboards; 

2) content knowledge (CK) - knowledge of the subject matter being taught to students, 
e.g. Estonian, mathematics, geography; 

3) pedagogical knowledge (PK) - knowledge of teaching methods, lesson planning, 
assessment and general classroom management skills; 

4) pedagogical content knowledge (PCK) - knowledge of how to teach particular content- 
based material to students; 

5) technological content knowledge (TCK) - knowledge of how to select and use different 
technologies to communicate particular content knowledge; 

6) technological pedagogical knowledge (TPK) - knowledge of using technology to 
implement different teaching methods; 

7) technological pedagogical content knowledge (TPACK) - knowledge of using 
technology to implement teaching methods for different types of subject matter content 
(Koehler & Mishra, 2009). 
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Figure 1: TPACK framework. 


Using TPACK as a framework for measuring teaching knowledge could potentially have 
an impact on the type of professional development experiences designed for in-service teachers 
(Schmidt et al., 2009). In recent years researchers have been measuring how effective their 
professional development efforts have been for the development of teachers’ TPACK (Graham 
et al., 2009; Guzey & Roehrig, 2009). 

One strategy to determine growth in TPACK over time is to use assessments before and 
after a special course or training program (Chai, Koh & Tsai, 2010; Hu & Fyfe, 2010; Hoffer & 
Grandgenett, 2012). In their study of pre-service teachers in an educational technology course 
in Singapore, Chai, Koh and Tsai (2010) concluded that participants made significant gains 
in CK, PK, TK, and most substantially in TPACK with fairly large effect sizes. Hu and Fife 
(2010) completed a similar study in an educational technology course in Australia. Post-course 
survey results indicated that the teachers’ confidence in their ability to connect their use of 
technology with content and pedagogy increased significantly. Hoffer and Grandgenett (2012) 
examined pre-service teachers’ TPACK across an 11-month training program and detected 
significant growth in the students’ TPACK throughout the study. Kurt, Mishra and Kocoglu 
(2013) conducted a survey of pre-service teachers in Turkey and the findings revealed that there 
was a Statistically significant increase in their TK, TCK, TPK and TPACK scores. Graham et 
al. (2009) studied the TK, TCK, TPK and TPACK of in-service teachers who participated in 
an intensive professional development program in a US university. The results indicated that 
the participants began and ended the course with the greatest level of confidence in their TK, 
followed by TPK, TPACK and finally TCK. 

There is evidence that teachers’ overall TPACK perceptions are influenced by certain 
knowledge domains of TPACK. Some studies (Chai, Koh & Tsai, 2010; Chai, Koh, Tsai & 
Tan, 2011) have revealed that pedagogical knowledge (PK) and technological pedagogical 
knowledge (TPK) have the biggest impact on the development of TPACK. Koh and Sing (2011) 
discovered strong positive correlations between pre-service teachers’ TPK, TCK and TPACK. 
In a qualitative study, Koh and Divaharan (2011) also found that pre-service teachers focused 
mostly on issues associated with TPK. 

So far research in this field is mostly based on short-term courses; thus, longitudinal 
studies are needed to examine the TPACK development of both pre-service and in-service 
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teachers across time (Hoffer & Grandgenett, 2012; Koh & Sing, 2011). Hoffer and Grandgenett 
(2012) add that triangulated study designs that span multiple years will help to not only better 
understand how TPACK develops, but also know which factors support and inhibit this growth. 
Moreover, TPACK studies have generally been reported for US teachers (e.g. Schmidt et al., 
2009; Archambault & Crippen, 2009; Graham et al., 2009). The effectiveness of TPACK-based 
training courses has not yet been reported in Estonia. 

Taking all this into consideration, a two-year in-service training program of educational 
technology (60 ECTS) was developed in Tallinn University Haapsalu College during the 
academic year of 2012/2013 to support in-service teachers to effectively embed ICT into their 
classroom teaching. The program consists of 17 different subjects and pedagogical practice. It 
is based on the Technological Pedagogical Content Knowledge (TPACK) framework developed 
by Punya Mishra and Matthew J. Koehler in 2006 and the National Educational Technology 
Standards for Teachers developed by ISTE (International Society for Technology in Education) 
in 2008. 

The aim of this study was to examine the perceived development of in-service teachers’ 
TPACK during the first year of their educational technology professional development. The 
key questions for this research were: 1) how do teachers’ perceptions of their TPACK levels 
change after participating in the training program for a year; 2) which of the seven knowledge 
domains of the TPACK framework develop more than others; 3) what is the impact of different 
knowledge domains on teachers’ TPACK perceptions. 


Methodology of Research 
General Background of Research 


The first academic year of the professional development training program designed in 
Haapsalu College included nine different courses (altogether 30 ECTS): Effective Computer 
Usage (5 ECTS), Teaching and Learning in the Digital Age (3 ECTS), Basics of Multimedia 
(3 ECTS), Digital Literacy (3 ECTS), Digital Media Production (4 ECTS), e-Society, Law 
and Security Issues in e-Learning (3 ECTS), Creating Digital Learning Resources (3 ECTS), 
Web-Based Learning Environments and Networks (3 ECTS), Educational Video Design 
and Implementation in the Classroom (3 ECTS). These courses provide in-service teachers 
with different knowledge domains of the TPACK framework. Content knowledge (CK) and 
pedagogical content knowledge (PCK) are the only domains not taught during the program as 
the participating teachers are considered experts in their subject matter. 

During the studies the participants learn to create and analyse different teaching resources 
(educational videos, presentations, learning activities) and lesson plans. The designed training 
program strives to adhere to the following instructional approaches and conditions: active 
engagement, authentic learning experiences, a variety of learning strategies, peer collaboration, 
sharing, support, reflection, etc. The training of in-service teachers takes place every other 
Friday and Saturday (altogether sixteen 8-hour training days in a semester). In addition to face- 
to-face meetings there is constant online communication through blogs, email, Facebook and 
Twitter. 

The present study is the first part of a longitudinal action research, which aims to examine 
the impact of the in-service training program of educational technology on teachers’ TPACK, 
especially the application of TPACK in the classroom and its effect on student learning. 

The practice of action research has been fairly common among researchers interested in 
teaching practices and teacher education. Action research appears to be a particularly effective 
method for studying and improving TPACK (Manfra & Bullock, 2013). Borthwick & Pierson 
(2008) stress that successful professional development efforts must resemble the iterative 
action research cycle: “encouraging teachers to begin any training by thinking about what they 
can learn from it and how, what they are to learn is situated in the work that they already do; 
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posing questions for how teaching and learning can improve; collaborating with peers and those 
more experienced to work to solve problems of practice; and evaluating and sharing findings 
with one another as part of an ongoing effort at collective improvement” (p. 211). The purpose 
of action research is to bring about change in specific contexts. It involves a cyclical process 
of action and reflection and a systematic approach to data collection and analysis (Manfra & 
Bullock, 2013). 

One key criterion that distinguishes action research from other types of research is that 
there is always someone directly involved in the situation who serves as researcher (Hinchey, 
2008). The author of this paper is both the head of the training program and one of the lecturers 
of educational technology in the training program. 


Sample of Research 


The sample of the pilot study was the group of in-service teachers who started their 
educational technology professional development program in Tallinn University Haapsalu 
College in January 2014. In-service teachers’ letters of motivation served as an entrance 
examination to help decide which teachers would be admitted to the training program. The 
candidates (n=33) were asked to evaluate their knowledge and skills in the field of educational 
technology and their motivation to participate in the program. All the selected participants 
(n=20) were in-service teachers with several years of teaching experience. Of the 20 in-service 
teachers enrolled in the training program, 17 participated in the study. One participant unenrolled 
from the program in the third week of the semester and two participants at the end of the first 
semester. The remaining 17 teachers completed all data collection instruments in the study. 

13 of the 17 participants worked as class teachers (primary school teachers), five were 
qualified as English teachers and two were teachers of Estonian. All the participants were 
female. The teachers ranged in age from 28 to 62 with a mean of 42 years. 


Instrument and Procedures 


The TPACK questionnaire developed by Schmidt et al. (2009) was selected to measure 
the in-service teachers’ self-assessments of their TPACK. Self-report measures, which ask 
participants to rate the degree to which they agree to a given statement, are one of the most 
frequently used methods to measure participants’ TPACK (Koehler, Shin & Mishra, 2012). 

The survey instrument was used with one modification — a module of creative / talent- 
based subjects (art, handicraft, music) was added to include all the subjects that Estonian class 
teachers generally teach in their classrooms. The final instrument contained 52 statements for 
measuring teachers’ self-assessments of the seven TPACK domains. All the statements were 
on a five-point Likert scale: 1) strongly disagree; 2) disagree; 3) neither agree nor disagree; 4) 
agree; 5) strongly agree. 

The online survey was administered to all the participants during the first meeting of 
the training program in January 2014 and at the end of the first academic year in January 2015. 
Before the pre-course survey the TPACK framework was introduced to the participants in the 
classroom, the purpose of the study was explained, and the in-service teachers were told that 
their participation in the study was voluntary. It took approximately 15-20 minutes to fill in the 
survey. 


Data Analysis 
In order to analyse the data collected, descriptive statistics were used. Means and standard 
deviations were calculated for each of the survey items and also for all the knowledge domains 


of TPACK. A paired samples t-test (also called repeated measures) was used to calculate 
differences between pre-course and post-course scores and to determine the likelihood that pre- 
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post differences were not due to chance. Cohen’s d was computed to determine effect size. In 
order to examine the impact of different knowledge domains on teachers’ TPACK perceptions, 
Pearson’s correlation coefficients were calculated between TPACK and all its constructs. 


Results of Research 


The present study set out to address the following two research questions first: How do 
teachers’ perceptions of their TPACK levels change after participating in the training program 
for a year and which of the seven knowledge domains of the TPACK framework develop more 
than others? 

Table 1 provides the means, standard deviations (SD) and t-test results for all the 
knowledge domains of TPACK. 


Table 1. In-service teachers’ TPACK development during the course. 


























Pre-course survey Post-course survey 
Knowledge Mean SD ear sD atl p Cohen’s d 
domains 
CK 3.34 0.81 3.54 0.87 0.20 .00* 0.24 
PK 4.01 0.59 4.19 0.47 0.18 01" 0.34 
TK 3.13 0.77 ~=— 3.61 0.71 0.48 .00* 0.65 
PCK 3.28 0.72 3.58 0.95 0.30 .00* 0.36 
TCK 2.94 0.77 = 3.27 0.92 0.33 .00* 0.39 
TPK 3.75 0.68 412 0.61 0.37 .04* 0.57 
TPACK 3.02 0.89 3.65 0.80 0.63 .00* 0.75 
* p<0.05 


Results from the pre-course survey indicated that the participants began the training 
program with the greatest level of confidence in their PK, followed by TPK and CK. The 
teachers were least confident about their TCK and TPACK. At the end of the first academic 
year the in-service teachers had the greatest level of confidence in their PK, followed by TPK 
and TPACK, but the lowest level of confidence in their TCK and CK. 

The comparison of self-assessment reports revealed significant development of the 
teachers’ technological knowledge (TK) and technological pedagogical content knowledge 
(TPACK). There was only limited growth in participants’ CK and PK, which is not surprising 
as content knowledge (CK) and pedagogical content knowledge (PCK) were not taught during 
the program. 

Paired t-tests conducted between all the knowledge domains of TPACK indicated the p 
< .05 level of significance for each pairing. The value of Cohen’s d was medium (between 0.5 
and 0.8) in case of TK, TPK and TPACK. 

The mean differences between the pre- and post-course scores were calculated for 
each statement. The next two tables present the impact of training on teachers’ technological 
knowledge (Table 2) and technological pedagogical content knowledge (Table 3) — the domains 
that were subject to the largest growth during the training program. 
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24| Table 2. The impact of training on teachers’ TK. 


























Statements Pre Post Change 
| know how to solve my own technical problems. 2.76 3.29 +0.53 
| can learn technology easily. 3.18 3.35 +0.17 
| keep up with important new technologies. 3.47 4.00 +0.53 
| frequently play around with the technology. 3.76 4.18 +0.42 
| know about a lot of different technologies. 247 3.24 +0.77 
| have the technical skills | need to use technology. 2.82 3.47 +0.65 
| have had sufficient opportunities to work with different technologies. 3.47 3.71 +0.24 





As can be seen from the chart, there is observable growth in the post-course survey 
results for all the statements, but particularly for statements 1, 3, 5 and 6. In the post-course 
test teachers’ self-assessments were the highest for statements 3 and 4, which shows teachers’ 
confidence in their ability to keep up with new technologies and interest in ‘tinkering’ with 


technology. 


The impact of training on teachers’ TPACK is also considerable. The post-course survey 
results show significant growth, particularly for statements 1, 4, 6, 7 and 8. In the post-course 
test teachers’ self-assessments were the highest for statements | and 8. Teachers are confident 
about their ability to combine mathematics lessons with appropriate technologies and teaching 
methods and they say they can choose technologies that enhance the content for a lesson. 


Table 3. The impact of training on teachers’ TPACK. 


























Statements Pre Post Change 
can teach lessons that appropriately combine mathematics, technologies, and teach- 3.08 3.92 +084 
ing approaches. 
| can teach lessons that appropriately combine literacy, technologies, and teaching 3.18 3.65  +0.47 
approaches. 
| can teach lessons that appropriately combine science, technologies, and teaching 3.08 3.54  +0.46 
approaches. 
can teach lessons that appropriately combine social studies, technologies, and teach- 2.85 3.62 +077 
ing approaches. 
I can teach lessons that appropriately combine creative/talent-based subjects, tech- 277 3.08  +0.31 
nologies, and teaching approaches. 
| can select technologies to use in my classroom that enhance what | teach, how | 3.18 3.88  +0.70 
teach, and what students learn. 
| can provide leadership in helping others to coordinate the use of content, technolo- 

: j soe 2.82 3.59 +0.77 
gies, and teaching approaches at my school and/or district. 
| can choose technologies that enhance the content for a lesson. 3.18 3.94 ——-+0.76 





In order to answer the third research question (What is the impact of different knowledge 
domains on teachers’ TPACK perceptions ?), pre-course and post-course correlation coefficients 


were calculated between TPACK and its knowledge domains (Table 4). 
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Table 4. Correlation coefficients between TPACK and its knowledge domains. 




















Knowledge domains Pre-course survey TPACK Post-course survey TPACK 
CK 0.51 0.46 
PK 0.32 0.20 
TK 0.66 0.57 
PCK 0.81 0.57 
TCK 0.69 0.74 
TPK 0.79 0.59 





Strong positive correlations were found between TK and TPACK (r=0.66), TCK and 
TPACK (r=0.69), TPK and TPACK (r=0.79), PCK and TPACK (r=0.81) in the pre-course 
survey. In the post-course survey TPACK was significantly correlated with TPK (r=0.59) and 
TCK (r=0.74). TK and PCK also had moderate positive correlations with TPACK as the results 
were Close to 0.60 (recommended by Fraenkel & Wallen, 2003). The correlations between CK, 
PK and TPACK were comparatively weaker. 


Discussion 


The preliminary findings of this study indicate overall strong growth in teachers’ 
perceptions of all the knowledge domains of TPACK, with the largest growth in teachers’ TK, 
TPK and TPACK. The significant growth in TK and TPACK makes sense as different courses 
during the program provided teachers with many opportunities to learn about new technologies 
and their implementation in the classroom setting. Researchers believe that teachers’ confidence 
in their TK might later help them to develop confidence in other knowledge domains (Graham et 
al., 2009). Technological knowledge is undoubtedly one of the foundations for ICT integration, 
and studies have shown that raising teachers’ technological skills increases the likelihood of 
them using ICT in the classroom (Hammond et al., 2011). 

The present study reveals that in-service teachers have the lowest level of confidence 
in their technological content knowledge (TCK) both in the pre- and post-course survey. This 
agrees well with the study of Graham et al. (2009). As most of the participants were primary 
school teachers, they might have been less confident than secondary school teachers in their 
ability to select and use different technologies to communicate particular content knowledge, 
especially as most of the teachers did not rate their content knowledge very highly either. 

The study also reveals significant correlations between the seven knowledge domains of 
TPACK, with the highest correlations between TPACK and TPK, and TPACK and TCK. These 
findings are consistent with the results of Schmidt et al. (2009) and Koh & Sing (2011). 

However, the present study has several limitations that should be considered when 
interpreting its results. Firstly, the duration of the study was limited to the first academic year of 
the training program. As the knowledge domains of TPACK might not develop at the same time 
and in the same way, TPACK should be examined at various points during the whole training 
program. 

Secondly, self-assessment reports may be prone to teachers under- or over-reporting 
their skills. Measuring TPACK using a self-report measure alone may be inadequate (Hoffer 
& Grandgenett, 2012) as participants tend to respond in ways that reflect positively on their 
abilities and knowledge. 

Finally, the development of TPACK was based purely on the in-service teachers’ self- 
reports. Therefore, it is unclear whether the professional development program has truly 
changed teachers’ practice. 
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Conclusions 


The value of any framework of technology integration lies in how it manages to influence 
pedagogical practice. Therefore, the evaluation of the in-service training program continues. 
The participating teachers will be asked to assess their TPACK once more, after completing the 
whole training program in December 2015. In addition, interviews with the participants will 
be conducted to examine what factors support and/or inhibit the development of their TPACK. 
Classroom observations of the in-service teachers will also be conducted and their lesson plans 
examined to evaluate the level of TPACK demonstrated in the classroom. Few studies address 
the effect of teacher professional development on student achievement, although experience 
around the world has shown that teacher training in the effective use of technology is the key 
determining factor for improved student performance. Hence, the author also plans to involve 
the students of the participating in-service teachers in the research. 

The findings of the present study are primarily significant for Tallinn University Haapsalu 
College as these help its program coordinators further develop and improve the training 
program of educational technology in order to better prepare in-service teachers to integrate 
technology in their classrooms. Furthermore, the findings have implications for the field of 
TPACK measurement and research. 

It is important to conclude with emphasizing that it is crucial to enable all teachers to be 
confident with technologies. Technologies can provide powerful tools for student learning, but 
their value depends on how effectively teachers use them to support instruction. 
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